A camera may produce radiometric measurements of the space within its field of view. It may periodically produce an image, or collection of these measurements. Each radiometric measurement may provide some information about the intensity, frequency and/or polarization of the electromagnetic waves traveling along a physical line, from a point on the line towards the camera at the time of the measurement. For example, a monochrome camera may produce measurements of the intensity of visible light traveling along a ray. A color camera may produce measurements R, G and B of the respective red, green and blue components of the visible light traveling along a ray.
Cameras may produce data in raster order, beginning data output at the upper left pixel, and progressing across the top row from left to right, then returning to the beginning of the next row, etc., until the bottom right pixel is reached. This sequence of pixels in raster order may he referred to as a stream of pixels, or a pixel stream. Each row of data can often be referred to as a scanline.
Stereo vision is the reconstruction of a three-dimensional (3D) structure in a 3D scene using two or more images of the 3D scene, each acquired from a different viewpoint. The images of the 3D scene may be obtained using multiple cameras or one moving camera. Two cameras are often used, termed binocular vision, which is similar to human vision through two eyes. With binocular vision, a stereo image is based on a left image as “seen” from a left camera (a perspective from the left) and a right image as “seen” from a right camera (a perspective from the right).
In image processing, more particularly computer vision, the term disparity refers to the difference in coordinates of similar features within left and right images of a stereo image. The disparity of features between the left and right images may be computed as a shift to the left (e.g., in pixels) of an image feature when viewed in the right image. If stereo images have lens distortion, or are not correctly aligned, image rectification may be performed to remove distortion and correct alignment of the images such that disparities exist only in the horizontal direction, Once rectified, stereo matching may be performed by a linear search for correspondence of features. For stereo matching, an algorithm may be used to scan the left and right images (i.e., process pixels of the images in a particular order) to search for matching image features. A pixel and one or more of its surrounding pixels in the left image are compared to all of the disparities in the right image by comparing corresponding pixel groups. Various metrics may be used for the comparisons. Using one of the metrics, the disparity with the best computed match is considered the disparity for the image feature. This minimum match score is an indication that the matched pixels correspond and hence the shift represents the disparity.
Stereo algorithms can be divided into those that can be done in one pass over the image data and algorithms that require multiple passes, One-pass algorithms tend to he the algorithms used for real-time applications due to their low computational cost and latency. These algorithms generally decide what the best disparity answer is for a pixel based upon a small amount of surrounding local image information. Multi-pass algorithms tend to aggregate information from all parts of the image and have tended to produce denser accurate results, but at considerable additional computational cost and latency.
In the drawings, the leftmost digit(s) of a reference number may identify the drawing in which the reference number first appears.